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The  goal  of  the  project  is  to  produce  an  authoring  system  to  permit  local 
experts  to  prepare  material  for  delivery  by  machine  at  the  work  site.  A 
system  to  permit  preparation  of  materials  for  the  Lincoln  Terminal  Sys- 
tem is  being  deployed  to  the  5th  Combat  Communications  Group  at  Robins 
APB.  A set  of  authoring  procedures  is  included  that  enables  authors  to 
prepare  lessons  on  the  methods  and  rationale  for  maintenance  of  elec- 
tronic gear,  and  on  various  other  subject  matters,  A second  revision  of 
these  authoring  procedures  is  undergoing  test.  Facilities  are  being 
readied  to  permit  local  generation  and  checkout  of  materials  with  a com- 
puter assist. 
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ARPA  AUTHORING  SYSTEM 


I.  INTRODUCTION 

A.  Problem  Statement 

The  goal  of  this  project  is  to  design,  deploy,  field  test,  and  evaluate  a system  to  enable 
locally  qualified  personnel  to  prepare  individualized  instructional  material  for  on-the-job  train- 
ing (OJT)  by  machine  delivery.  O.IT  is  the  only  appropriate  type  of  training  for  many  opera- 
tional, maintenance,  and  repair  tasks  because  these  tasks  are  unique  to  particular  pieces  of 
equipment  and  to  the  environment  in  which  the  equipment  is  used. 

Currently,  O.IT  is  conducted  by  qualified  operators  and  technicians  who  instruct  and  super- 
vise novice  personnel,  most  frequently  on  a one-to-one  basis.  These  same  operators  and  tech- 
nicians are  also  responsible  for  equipment  maintenance,  repair,  readiness,  and  operations. 

As  a consequence,  training  is  often  unavailable  and,  at  best,  inefficient.  Capturing  the  exper- 
tise of  these  technicians  and  operators  in  suitable  form  for  automated  delivery  could  provide 
continuous  availability  of  training,  decrease  the  utilization  of  expert  personnel  for  one-to-one 
instruction,  further  standardize  training,  and  increase  the  overall  effectiveness  of  O.IT. 

While  the  feasibility  of  machine  delivery  of  instruction  at  the  work  site  has  been  demon- 
strated with  the  Lincoln  Terminal  System  (I.TS),  the  acquisition  of  appropriate  instructional 
material  remains  a problem.  Professional  technical  instructors  or  writers  are  not  suited  to 
the  task  of  generating  such  material  because  they  do  not  generally  have  the  extensive  local  op- 
erational experience  and  up-to-date  technical  knowledge  that  are  required  to  design  materials 
with  the  necessary  precision  and  relevance  of  content.  Operational  personnel  do  not  have  the 
requisite  authoring  skills.  The  goal  of  this  project  ’S  to  develop  procedures  and  facilities  that 
will  compensate  for  this  lack  of  authoring  experience,  and  will  enable  locally  qualified  opera- 
tional experts  to  prepare  materials  that  will  meet  the  needs  of  OJT. 

li.  Background 

Under  the  sponsorship  of  the  Air  Force  Communications  Service  (AFCS),  Lincoln  Labora- 
tory has  recently  carried  out  feasibility  tests  of  machine-supported  O.IT  at  the  5th  Combat  Com- 
munications Group  at  Robins  AFB.  The  5th  CCG  was  chosen  on  the  recommendation  of  AFCS 
because  the  training  problems  at  that  site  are  representative  of  those  elsewhere  in  the  command 
and  are  particularly  acute.  These  tests  demonstrated  that  novice  (3-level)  personnel  at  the 
5th  CCG  were  able  to  execute  extensive  maintenance  procedures  with  few  errors  and  little  out- 
side help  when  the  procedures  were  presented  on  the  LTS-3S.  (This  unit  is  part  of  the  Lincoln 
Terminal  System;  i combines  aud;  ->/ visual  presentation  of  information  with  computerized  man- 
agement of  learning  in  a stand-alone  machine.)  Although  the  trainees  were  not  presented  with 
the  full  rationale  behind  the  procedures,  they  acquired  hands-on  experience  which  was  impor- 
tant as  a basis  for  further  skill  knowledge  training  and  considerably  speeded  up  their  readiness 
for  independent  line  work.  These  findings  have  been  applied  by  the  5th  CCG  in  implementing  an 
interim  training  program  using  the  LTS-3S  and  supplemental  classroom  instruction. 

Our  experience  with  the  5th  CCG  lias  afforded  us  a realistic  appraisal  of  their  OJT  require- 
ments and,  with  the  continuing  cooperation  of  AFCS,  a suitable  environment  for  the  experimental 
development  and  evaluation  of  an  auth  ring  system.  Accordingly,  a substantial  amount  of  time 
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during  the  past  several  months  has  been  devoted  to  laying  groundwork  with  the  5th  CCG,  Hqs. 
AFCS,  and  TACCOMMAREA  (the  headquarters  to  which  the  5th  CCG  reports)  for  a cooperative 
program  of  work  directed  at  the  problem  ot  acquiring  instructional  materials  for  OJT. 

II.  AUTHORING  PROCEDURES  DEVELOPMENT 

A.  LTS  Lesson  Authoring  Procedures 

This  project  tests  the  hypothesis  that  lesson  authoring  can  be  redjced  to  a procedure  which 
operational  personnel  without  prior  authoring  experience  can  follow  to  produce  an  adequate  les- 
son. The  lesson  type  of  present  concern  is  called  a Task  Procedure  and  Interpretation  (TPI), 
which  is  a documented,  step-by~step  description  of  how  each  task  of  interest  is  to  be  performed 
and  why.  It  provides  procedural  information  and  also  the  rationale  for  each  step  in  terms  of 
what  each  step  accomplishes  in  the  task.  It  provides  enough  detail  so  that  the  trainee,  with  re- 
cess to  the  LTS  (or  other  lesson  delivery  system),  can  learn  to  perform  the  task  and  understand 
what  he  is  doing  with  only  minimal  supervision. 

The  techniques  of  authoring  TPIs  were  derived  from  those  general  principles  of  instruc- 
tional design  that  appeared  relevant  to  the  specific  purposes  of  TPIs.  These  techniques  are  now 
incorporated  in  a 30-page  how-to-do-it  manual  with  a set  of  worksheets  and  an  example  of  a 
validated  TPI.  The  manual  is  itself  a TPI  designed  for  technical  experts  who  have  not  previously 
written  teaching  materials  and  for  whom  writing  teaching  materials  will  probably  be  a second- 
ary and  short-lived  duty.  Rather  than  present  many  rules  at  once  for  the  author  to  keep  in  mind 
while  he  is  writing,  he  is  given  only  a few  rules  to  incorporate  at  each  stage  in  the  process  jf 
lesson  development.  Each  step  or  rule  is  explained  in  detail,  and  a brief  rationale  for  the  rule 
is  given.  Illustrations  of  the  implementation  of  each  rule  are  referenced  in  a sample  TPI  that 
accompanies  the  manual. 

In  the  first  section  of  Authoring  Procedures,  the  author  is  asked  to  divide  his  task  into  sev- 
eral major  parts,  then  outline  the  actions  or  steps  which  comprise  each  part.  Next,  he  is  asked 
to  design  the  logic  flow  of  hi s TPI  and  draw  a flowchart  for  it.  lie  begins  writing  frames  by  be- 
ing asked  to  write  several  introductory  frames  following  structured  directions  and  models.  The 
purpose  of  this  is  to  ease  him  into  the  writing  task  painlessly.  Then  he  drafts  and  refines  the 
contents  of  his  other  frames.  He  is  given  guidelines  to  follow  regarding  the  level  of  explanation, 
the  amount  of  material  in  each  frame,  the  types  of  visuals  which  should  work  best  for  his  pur- 
poses, the  length  and  content  of  the  audio  messages  in  each  frame,  and  the  logic  instructions 
which  must  be  written  for  the  LTS  in  each  frame. 

R.  Training  Coordinator 

The  process  of  TPI  preparation  includes  a training  coordinator  role,  which  will  presumably 
be  instituted  at  each  I.TS-supported  training  site.  The  training  coordinator  would  have  at  least 
three  major  functions;  (1)  supporting  l.TS  authors,  (2)  designing  training  programs  for  trainees, 
and  (3)  collecting  data  for  long-range  project  evaluation. 

A second  manual  lias  been  prepared  to  guide  the  training  coordinator  in  the  first  function  - 
that  of  supporting  l.TS  authors.  This  manual  provides  the  necessary  guidance  for  selecting 
TPI  authors  and  tasks,  counseling  and  editing  for  authors  during  the  actual  writing  stages  of 
TPI  preparation,  conducting  TPI  tryouts  and  collecting  revision  data,  coordinating  the  typing 
and  filming  of  TIM  manuscripts,  repotting  on  the  successes  and  failures  of  the  authoring  experi- 
ment, and  suggesting  revisions. 
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C.  Test  of  Authoring  Procedures  to  Date 

Two  individuals  from  Lincoln  have  been  involved  in  producing  TPIs  as  «t  formative  evalua- 
tion of  the  Authoring  Procedures  under  development.  The  first  person  is  a photographer  who 
operates  the  LTS  fiche  production  facility.  He  has  completed  (ready  for  final  artwork)  a TP1 
for  producing  a copy  from  a master  fiche.  The  second  person  is  an  Engineering  Assistant  in- 
volved in  LTS  design  and  implementation  who  is  preparing  a TP1  on  setup  of  the  LTS-3S  Re- 
cording Console  for  production  of  modulated  or  unmodulated  circles  for  test  fiche.  This  effort 
is  about  50-percent  complete. 

Discussion  and  extensive  investigations  were  conducted  with  qualified  experts  and  training 
personnel  in  the  5th  CCG  to  establish  an  adequate  sample  of  authoring  activity  to  aid  in  the  test- 
ing and  revision  of  authoring  materials  and  facilities.  At  this  point,  eight  efforts  are  in  vari- 
ous stages  of  progress: 

(t)  A TPI  on  one  of  the  performance  checks  for  the  AN/TRC-97A  is  about 
30-percent  complete,  and  two  TPIs  for  bench  alignment  are  in  the 
planning  stages. 

(2)  Three  TPIs  for  setting  up  conference  calls  on  the  TCC-30  telephone 
switching  van  have  been  outlined,  and  one  is  about  75-percent  complete. 

(3)  A TPI  on  camouflage  techniques  is  about  50-percent  complete. 

(4)  Work  has  begun  on  lessons  on  the  operation  of  the  Control  Head  for  the 
TSC-60  IIP  radio  set.  The  Control  Head  permits  remote  operation  of 
the  radio  set,  including  such  functions  as  frequency  selection  and  timing. 

(5)  A TPI  is  being  written  for  the  DAC-8  of  the  TSC-62  van.  The  DAC-8 
provides  checkout  and  control  of  TTY  circuits  in  the  communications 
system. 

(6)  A TPI  on  the  Astreon  409  for  the  TSC-6?  ch  control  van  is  being 
planned. 

(7)  At  least  one  area  of  teletype  opera’.  - vi  oe  chosen  for  lesson 
preparation. 

(8)  Instruction  in  Systems  Interface  Mi  iagement  \vi!'  be  outlined,  and 
certain  materials  will  be  produced  in  this  area. 

Topic  (8)  goes  somewhat  beyond  the  scope  of  training  material  addressed  by  the  Authoring 
Procedures.  This  effort  will  provide  valuable  feedback  in  helping  design  author  guidance  in 
preparing  lesson  material  of  a more  conceptual  nature  than  that  addressed  by  our  present  Au- 
thoring Procedures. 

The  feedback  from  ongoing  authoring  activities  has  been  most  encouraging.  Primarily, 
there  is  every  indication  that  the  TPI  Authoring  Procedure  is  written  at  an  appropriate  level 
and  covers  most  of  the  questions  that  occur  in  the  authoring  process,  and  that  the  example  which 
accompanies  the  manual  has  proven  to  be  a valuable  guide  for  novice  authors.  Several  deficien- 
cies have  become  apparent.  In  the  first  edition,  we  do  not  give  sufficient  emphasis  to  the  im- 
portance of  interaction  and  monitoring  of  student  behavior  - which  is  a unique  capability  offered 
by  a computer-augmented  delivery  medium  like  LTS.  Somewhat  related  to  this  is  a failure  to 
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clearly  emphasize  that  other  modes  of  interaction  than  those  suggested  in  the  Authoring  Proce- 
dure are  certainly  feasible,  and  may  indeed  be  desirable.  Another  is  the  relationship  of  frames 
to  layout  on  the  actual  fiche,  and  the  general  idea  of  access  to  a single  frame  from  several  points 
within  a TPI.  These  and  other  issues  are  certainly  being  considered  in  the  iterative  revision  of 
the  Authoring  Procedure. 

With  a moderate  amount  of  huma-i  guidance,  operational  personnel  have  bee”  able  to  proceed 
with  little  hesitancy.  The  three  partially  completed  examples  indicate  that  the  .•>  . procedures 
can  be  laid  out  in  a concise  and  orderly  manner,  with  a good  understanding  of  where  corrective 
or  supporting  material  ought  to  be  included.  The  one  completed  example  is  quite  good  and,  with 
minor  additions,  should  make  another  model  example. 

III.  AUTHOR  SUPPORT  FACILITIES  DEVELOPMENT 

A.  L.TS-5  Simulator 

Work  has  been  initiated  on  the  LTS-5  simulator  that  runs  on  a PDP-8/Floppy-Disk  System 
which  will  simulate  the  operation  of  the  new  LTS-5.  The  purposes  of  the  simulator  are  to: 

(1)  provide  authors  with  a means  of  checking  modules  prior  to  committing  the  logic  to  fiche,  and 

(2)  provide  students  with  a virtual  LTS-5  to  be  used  in  conjunction  with  lesson  material  in  note- 
book form.  In  the  simulator,  disk  files  are  used  to  store  the  frame  logic,  Th”  projected  date 
of  completion  of  the  simulator  system  and  all  related  documentation  is  mid -August  1976. 

B.  LTS-5  Delivery  Unit 

The  hardware  portion  of  the  original  proposal  called  for  the  construction  of  one  LTS-5  and 
the  modification  of  one  LTS-3S  to  make  it  a functional  equivalent  of  an  LTS-5.  It  has  been  de- 
termined that  it  will  be  possible  to  construct  two  LTS-5' s within  the  constraints  of  available 
manpower  and  funds.  The  LTS-5  model  has  been  developed  with  support  from  the  Bureau  of 
Mines  (BoM)  over  the  last  12  months.  Under  the  provision  of  the  BoM  contract,  a production 
prototype  of  the  LTS-5  will  be  constructed  during  FY  76T.  We  propose,  therefore,  to  construct 
at  the  same  time  two  additional  such  units  for  the  DARPA  program.  Ordering  of  parts  for  the 
two  DARPA  units  is  complete,  most  are  now  on  hand,  and  construction  is  under  way.  The  BoM 
contract  also  calls  for  the  upgrading  of  the  software  package  used  with  the  LTo  . This  rede- 
sign work  is  essentially  complete  and  will  be  available  for  use  with  the  DARPA  equipments. 

IV.  APPLICATIONS  AND  PLANS 

A.  Further  Considerations 

This  project  is  part  of  a larger  effort  to  improve  the  means  for  obtaining  materials  for  in- 
dividualized instruction.  It  is  oriented  toward  the  subject  matter  expert  being  the  author,  in 
contrast  to  the  instructional  expert.  More  specifically,  the  kind  of  subject  matter  expert  is  one 
who  must,  as  part  of  his  operational  responsibilities,  indoctrinate  novice  personnel  in  work 
procedures. 

TPIs  are  a specific  type  of  lesson  appropriate  to  O.IT  implemented  in  accordance  with  gen- 
eral principles  of  instructional  design.  Applied  to  technician  training,  they  are  characterized 
by  implicitly  stated  behavioral  objectives,  explicit  checks  on  performance,  a lesson  stiucture 
which  follows  task  structure,  and  reliance  on  reference  documents  rather  than  task  memorization. 
Other  types  of  learning  present  a different  set  of  requirements.  For  operator  training  - such  as 
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piloting,  communicating,  or  air  traffic  control  - a prototypical  lesson  would  place  a greater 
emphasis  on  memorization.  Training  materials  would  therefore  support  task  rehearsal  until 
rapid  and  accurate  performance  is  achieved.  We  anticipate  defining  and  developing  authoring 
procedures  for  a prototypical  lesson  of  this  type  in  addition  to  the  TP1.  Support  of  the  TPI 
sometimes  requires  a rather  elaborate  interpretation  of  procedure  in  conceptual  terms,  and 
task  structure  is  not  a valid  basis  for  the  design  of  such  material.  In  that  case,  selection  of 
behavioral  objectives  would  play  a greater  part  in  lesson  structuring,  and  the  modes  of  interac- 
tion would  differ  significantly.  The  design  methods  and  procedures  developed  for  in-school 
training  would  be  appropriate. 

Current  assessments  of  the  cost  effectiveness  of  preparation  of  individualized,  computer- 
based  instruction  are  generally  not  encouraging.  The  difficult  part  of  authoring  is  to  determine 
behavioral  objectives  and  to  design  lesson  material  so  that  it  explains  basic  concepts.  The  va- 
riety, complexity,  and  novelty  of  concepts  require  a sophisticated  approach  to  lesson  design. 
When  there  is  a requirement  to  prepare  materials  for  individual  delivery  [Individually  Pro- 
grammed Instruction  (IPI)l,  with  or  without  diagnostic  and  remedial  aids  [Computer-Based  In- 
struction (CHI)],  the  establishment  and  maintenance  of  a team  of  skilled  a>  thors  are  major  and 
costly  undertakings.  These  conditions  prevail  in  the  school  environmer...  The  advent  of  the  LTS 
has  made  it  fruitful  for  the  first  time  to  give  serious  consideration  to  applying  these  new  methods 
to  the  support  of  O.IT.  Somewhat  surprisingly,  the  problems  of  authoring  seem  less  severe  for 
job- related  materials.  The  prognosis  for  the  success  of  the  TPI  approach  to  development  of 
materials  appears  to  be  good. 

In  O.IT,  the  author's  basic  rationale  is  usually  the  work  procedure  itself;  the  primary  ob- 
jectives are  already  known,  and  the  operational  expert  as  author  is  intimately  familiar  with  the 
subject  matter.  Thus,  a procedure  can  be  developed  for  construction  of  lessons,  and  extensive 
training  of  authors  can  be  obviated  and  the  work  distributed  among  the  experts  best  suited  to 
each  area.  Furthermore,  the  content  in  the  form  of  work  procedures  is  largely  agreed  upon, 
whereas  the  training  needs  and  requisite  concepts  underlying  technical  training  in  schools  are 
more  controversial.  These  conditions  favor  more  orderh  and  economic  development  of  ma- 
terials for  the  work  environment  than  for  the  school. 

The  requirement  to  try  out  the  authoring  procedures  in  an  operational  environment  in  order 
to  establish  the  validity  of  the  methods  appears  more  important  than  originally  conceived.  The 
authoring  of  materials  is  a chore.  Dividing  up  the  work  is  an  obvious  advantage  afforded  by  the 
procedural  approach.  This  consideration  seems  overshadowed,  however,  by  the  motivational 
advantages.  In  this  situation,  both  the  author  and  his  commander  perceive  the  authoring  task  as 
part  of  a solution  to  an  immediate  and  important  problem.  Thus,  in  the  5th  C'CG,  the  l.TS  has 
been  shown  to  reduce  the  training  load  on  operational  experts,  end  this  is  the  basis  for  the  wide- 
spread interest  in  trying  out  the  authoring  procedures.  If  the  investment  in  lessons  by  the  ex- 
pert translates  directly  into  improved  work  performance,  it  seems  much  more  ’’kely  that  the 
benefits  of  the  technology  will  be  assured  in  the  long  run. 

B.  Test  Plan 

The  present  activity  supports  training  operations  at  the  5th  CCG  on  a small  but  useful  scale. 
Moau'.cs  of  instructional  materials  are  being  prepared  by  the  new  authoring  procedures,  and 
"format;  evaluation  consists  of  training  coordinator  and  peer  review.  The  materials  will  be 
evaluated  in  terms  of  their  effect  on  work  performance  and,  in  this  manner,  a rough  "summative" 
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evaluation  of  each  author  product  will  he  carried  out.  Technical  services  for  cutting  and  testing 
frame  logic  and  summarizing  student  data  will  be  installed  and  tried  out  in  November  1976.  This 
activity  will  serve  as  a test  bed  to  further  refine  and  expand  the  author  procedures  and  the  fa- 
cilities. The  process  of  improvement  will  continue  until  approximately  1 January  1977. 

There  is  a proposal  to  upgrade  training  in  the  5th  CCG  and  other  units  that  would  require  a 
very  substantial  initial  investment  in  manpower  to  prepare  materials.  It  would  hove  to  lie  fol- 
lowed either  by  a procurement  of  computer-supported  instructional  hardware  such  as  the  I.TS-5, 
or  by  provision  of  substantial  instructional  staff.  It  has  boon  suggested  that  a pilot  project  be 
initiated  instead,  a precursor  that  sets  the  pace  In  a few  critical  areas.  Either  a large-  or 
small-scale  effort  would  afford  opportunity  for  a subsequent  full-scale  field  test  of  the  author- 
ing procedures  and  facilities.  A plan  to  incorporate  the  authoring  test  in  the  larger  training 
effort  is  being  negotiated  at  present  with  the  Air  F'orce  Communications  Service. 
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